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What is claimed is: 



1 1. A method of reading data comprising the steps of: 

2 receiving a request for a stripe of erasure coded data stored across a 

3 plurality of storage devices, the stripe comprising stripe blocks; 

4 sending read messages to at least a quorum of the storage devices; 

5 receiving at least the quorum of reply messages from the storage 

6 devices, the quorum of the reply messages including at least a minimum 

7 number of the stripe blocks needed to decode the stripe of erasure coded 

8 data, the quorum meeting a quorum condition of a number such that any 

9 two selections of the number of the stripe blocks intersect in the minimum 

10 number of the stripe blocks; and 

1 1 decoding the stripe of erasure coded data from at least the minimum 

12 number of the stripe blocks, thereby forafiing the data. 

1 2. The method of claim 1 wherein each of the reply messages within the quorum 

2 indicates that there is no pending write for the stripe block stored on the storage 

3 device associated with the reply message. 

1 3. The method of claim 1 wherein each of the reply messages within the quorum 

2 indicates that the stripe block associated with the reply message has a timestamp 

3 that matches other timestamps associated with other reply messages within the 

4 quorum. 

1 4. The method of claim 1 wherein a coordinator performs the steps. 

1 5. The method of claim 4 wherein the coordinator comprises one of the storage 

2 devices. 

1 6. The method of claim 5 wherein the coordinator effectively sends one of the 

2 read messages to itself. 

1 7. The method of claim 6 wherein the coordinator effectively receives one of the 

2 reply messages from itself. 
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18. A method of reading data comprising the steps of: 

2 receiving a request for a stripe of erasure coded data stored across a 

3 plurality of storage devices, the stripe comprising stripe blocks which 

4 comprise a first number of data blocks and a second number of parity 

5 blocks; 

6 sending read messages to the storage devices; 

7 receiving at least a quorum of reply messages from the storage devices 

8 which indicate that there is no pending write for the stripe block stored on 

9 the storage device, the quorum of the reply messages including at least the 

10 first number of the stripe blocks, the quorum comprising the first number 

1 1 plus a half of the second number; and 

12 decoding the stripe of erasure coded data from the first number of the 

13 stripe blocks, thereby forming the first number of the data blocks. 

1 9. The method of claim 8 wherein the quorum of the reply messages includes 

2 validation timestamps which match. 

1 10. The method of claim 8 wherein a coordinator performs the steps. 

1 11. The method of claim 10 wherein the coordinator comprises one of the storage 

2 devices. 

1 12. The method of claim 1 1 wherein the coordinator effectively sends one of the 

2 read messages to itself. 

1 13. The method of claim 12 wherein the coordinator effectively receives one of 

2 the reply messages from itself. 

1 14. The method of claim 8 wherein the coordinator is not one of the storage 

2 devices upon which the stripe of erasure coded data is stored. 

1 15. The method of claim 8 further comprising the step of identifying a group of 

2 the storage devices as targets. 
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1 16. The method of claim 15 wherein the step of identifying the targets randomly 

2 picks the targets. 

1 17. The method of claim 15 wherein each of the query messages sent to the targets 

2 identifies the storage device as one of the targets. 

1 18. The method of claim 17 wherein the reply messages from the targets include 

2 the stripe blocks. 

1 19. The method of claim 8 wherein the storage devices comprise a distributed 

2 storage system. 

1 20. The method of claim 19 wherein the distributed storage system comprises a 

2 quantity of the storage devices. 

1 21. The method of claim 20 wherein the quantity of the storage devices 

2 corresponds to the first number of the data blocks plus the second number of the 

3 parity blocks. 

1 22. The method of claim 20 wherein the quantity of the storage devices exceeds 

2 the first number of the data blocks plus the second number of the parity blocks. 

1 23. The method of claim 22 further comprising the step of identifying the storage 

2 devices upon which the stripe of erasure coded data is stored. 

1 24. The method of claim 8 wherein each of the storage devices comprises a log, 

2 wherein the log comprises log entries of each successful write of data, the log 

3 entries comprising a stripe indicator, a write timestamp, and a physical location of 

4 the stripe block on the storage device. 

1 25. The method of claim 8 wherein the read messages include a stripe indicator. 

1 26. The method of claim 8 wherein the reply messages include a stripe indicator. 
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1 27. A method of writing data comprising the steps of: 

2 generating a timestamp; 

3 encoding the data as a stripe of erasure coded data, the stripe 

4 comprising stripe blocks; 

5 sending a query message including the timestamp to each of a plurality 

6 of storage devices; 

7 receiving a query reply message from each of at least a first quorum of 

8 the storage devices indicating that the timestamp is later than a pending 

9 write timestamp and, if a version of the stripe block exists on the storage 

10 device, indicating that the timestamp is later than a validation timestamp 

1 1 for the version of the stripe block; 

12 sending a write message to each of the storage devices including the 

13 timestamp and the stripe block destined for the storage device; and 

14 receiving a write reply message from each of at least a second quorum 

15 of the storage devices indicating that the timestamp is later than the 

16 validation timestamp if the validation timestamp exists, that the timestamp 

17 is no earlier than the pending write timestamp, and that the stripe block 

18 was successfully stored, the first and second quorums each meeting a 

19 quorum condition of a number such that any two selections of the number 

20 of the stripe blocks intersect in a minimum number of the stripe blocks 

21 needed to decode the stripe of erasure coded data. 

1 28. A method of writing data comprising the steps of: 

2 generating a timestamp; 

3 encoding the data as a stripe of erasure coded data, the stripe 

4 comprising stripe blocks which comprise a first number of data blocks and 

5 a second number of parity blocks; 

6 sending a query message including the timestamp to each of a plurality 

7 of storage devices; 

8 receiving a query reply message from each of at least a first quorum of 

9 the storage devices indicating that the timestamp is later than a pending 

10 write timestamp and, if a version of the stripe block exists on the storage 

1 1 device, indicating that the timestamp is later than a validation timestamp 
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12 for the version of the stripe block; 

13 sending a write message to each of the storage devices including the 

14 timestamp and the stripe block destined for the storage device; and 

15 receiving a write reply message from each of at least a second quorum 

16 of the storage devices indicating that the timestamp is later than the 

17 validation timestamp if the validation timestamp exists, that the timestamp 

18 is no earlier than the pending write timestamp, and that the stripe block 

19 was successfully stored, the first and second quorums each meeting a 

20 quorum condition of a third number such that the third number is at least 

21 the first number plus half of the second number. 

1 29. A computer readable memory comprising computer code for implementing a 

2 method of reading a stripe of erasure coded data, the method of reading the stripe 

3 of erasure coded data comprising the steps of: 

4 receiving a request for a stripe of erasure coded data stored across a 

5 plurality of storage devices, the stripe comprising stripe blocks; 

6 sending read messages to at least a quorum of the storage devices; 

7 receiving at least the quorum of reply messages from the storage 

8 devices, the quorum of the reply messages including at least a minimum 

9 number of the stripe blocks needed to decode the stripe of erasure coded 

10 data, the quorum meeting a quorum condition of a number such that any 

1 1 two selections of the number of the stripe blocks intersect in the minimum 

12 number of the stripe blocks; and 

13 decoding the stripe of erasure coded data from at least the minimum 

14 number of the stripe blocks, thereby forming the data. 

1 30. The computer readable memory of claim 29 wherein each of the reply 

2 messages within the quorum indicate that there is no pending write for the stripe 

3 block stored on the storage device associated with the reply message. 

1 31. The computer readable memory of claim 29 wherein each of the reply 

2 messages within the quorum indicate that the stripe block associated with the 

3 reply message has a timestamp that matches other timestamps associated with 

4 other reply messages within the quorum. 
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1 32. A computer readable memory comprising computer code for implementing a 

2 method of writing a stripe of erasure coded data, the method of writing the stripe 

3 of erasure coded data comprising the steps of: 

4 generating a timestamp 

5 encoding the data as a stripe of erasure coded data, the stripe 

6 comprising stripe blocks; 

7 sending a query message including the timestamp to each of a plurality 

8 of storage devices; 

9 receiving a query reply message from each of at least a first quorum of 

10 the storage devices indicating that the timestamp is later than a pending 

1 1 write timestamp and, if a version of the stripe block exists on the storage 

12 device, indicating that the timestamp is later than a validation timestamp 

13 for the version of the stripe block; 

14 sending a write message to each of the storage devices including the 

15 timestamp and the stripe block destined for the storage device; and 

16 receiving a write reply message from each of at least a second quorum 

17 of the storage devices indicating that the timestamp is later than the 

18 validation timestamp if the validation timestamp exists, that the timestamp 

19 is no earlier than the pending write timestamp, and that the stripe block 

20 was successfully stored, the first and second quorums each meeting a 

21 quorum condition of a number such that any two selections of the number 

22 of the stripe blocks intersect in a minimum number of the stripe blocks 

23 needed to decode the stripe of erasure coded data. 
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